Skip to content

fix(code): ISS-5963 — zero-byte plan.json, absent-workspace fail-open, and a per-command predicate in detect_spurious_complete - #189

Merged
mikeangstadt merged 1 commit into
mainfrom
fix/iss-5963-spurious-complete-empty-plan-and-fail-open
Aug 12, 2026
Merged

fix(code): ISS-5963 — zero-byte plan.json, absent-workspace fail-open, and a per-command predicate in detect_spurious_complete#189
mikeangstadt merged 1 commit into
mainfrom
fix/iss-5963-spurious-complete-empty-plan-and-fail-open

Conversation

@mikeangstadt

@mikeangstadt mikeangstadt commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Summary

ISS-5963 — Layer 1 of the false-COMPLETED PLAN loop. detect_spurious_complete in plugins/code/scripts/run-loop.sh is the upstream guard for a run that claims COMPLETE without producing the artifact it existed to produce.

What c3a4305 already landed (merged in #186, do not re-review): it inverted the early return so a --prd run with no plan.json is flagged PLAN_MISSING_AT_COMPLETION instead of waved through, scoped it by prd_file so EXECUTE was not swept in, kept the AWAITING_USER hard stop outranking the new branch, and added 6 tests.

Three verified gaps remained. This PR closes them.

1. A zero-byte plan.json was still reported as success

The ticket's own headline evidence is a 0-byte artifact, and that exact input still passed. [[ -f ]] is true on an empty file, so the run fell through the missing-plan branch entirely; jq then yielded nothing, pending_count fell back to 0, and the detector returned {} — clean. A file that exists is not a file that was produced.

classify_plan_artifact now returns missing | empty | unparseable | present, and the first three are all treated as "no plan was written". The emitted subcode stays PLAN_MISSING_AT_COMPLETION (consumers already key off it); only the message distinguishes the three.

2. An absent workspace failed CLOSED

With no workdir, the detector previously reported PLAN_MISSING_AT_COMPLETION. But the artifact was not produced and the workspace no longer exists are different facts, and only the first is evidence of spuriousness. Live-exit and boot-recovery reclaim the temp workdir immediately after finalization, so adjudicating a run whose directory is already gone flips a genuine success to FAILED with no re-run that repairs it — the same BLOCKING issue Layer 2 (#4806, symphony-alpha) hit and fixed by failing open. A missing workdir now means "cannot judge", so it judges nothing (AC3).

3. REQUEST_CHANGES was not modelled; EXECUTE was excluded only by accident

The predicate was "was a --prd passed?". That is a proxy, and it fails both ways: REQUEST_CHANGES — whose result bundle also declares plan.json required — was invisible to the guard, while EXECUTE was excluded only because it happens not to carry a PRD, not because anything said so.

run_owes_plan_json now reads the command (CLOSEDLOOP_COMMAND, or the --prompt spelling like execute-prompt, normalized):

  • PLAN, REQUEST_CHANGES → owe plan.json.
  • EXECUTE and the rest → excluded by name. EXECUTE's required artifact is execution-result.json, written only after a successful commit and push, so a legitimate no-changes run ends without it; a blanket "required artifact missing ⇒ spurious" rule would fail every one of those. There is a fixture for exactly that run (AC4's warning).
  • Unknown / empty command → falls back to the --prd proxy, i.e. today's behaviour. An older desktop sending nothing and a newer one sending a command this release has never heard of both degrade instead of crashing or blocking (AC5, both directions).

Left open, deliberately: REQUEST_CHANGES is enforced only on the not-produced-at-all axis. The harness seeds plan.json before an amend, so presence proves nothing there — this PR catches "no plan at all" and "seeded file still zero bytes", but not "the amend ran and produced nothing", which needs a pre-run baseline the detector is not given. That is called out in the code comment rather than guessed at, and the remainder of AC2 is reported open on the ticket.

Test plan

bash plugins/code/scripts/tests/test_spurious_complete.sh19 pass, 0 fail (6 pre-existing + 13 new), following the existing file's style.

Each gap was proven counterfactually — production line reverted, suite re-run, failure observed, line restored:

reverted result
classify_plan_artifact reduced to the old [[ -f ]] check 4 red — zero-byte, malformed-JSON, whitespace-only, and the seeded-zero-byte REQUEST_CHANGES case each returned {}, i.e. reported the run clean
the [[ ! -d "$workdir" ]] fail-open guard removed 1 red — a deleted workspace returned PLAN_MISSING_AT_COMPLETION, the genuine-success-to-FAILED flip
run_owes_plan_json reduced to [[ -n "$prd_file" ]] 4 red — both REQUEST_CHANGES cases went invisible ({}), and EXECUTE-with-a-PRD plus execute-prompt were falsely flagged

Also run: uv run --group dev ruff check . → All checks passed. bash -n clean on both changed scripts. uv run --group dev pytest plugins/ → 2103 passed, 3 skipped, exit 0. CI green on all five checks (Plugin Version Bump, Lint, Type Check, Tests, TypeScript (design-inventory)). (shellcheck -S error reports SC1073/SC1072 on run-loop.sh line ~2222 — verified identical on origin/main, pre-existing and untouched.)

The test file now clears ambient CLOSEDLOOP_COMMAND/PRD_FILE before sourcing, so the two-argument call sites cannot have their per-command branch decided by the shell that happens to be running the suite.

Breaking changes

None — additive and internal to the detector. The subcode contract is unchanged. Every newly-flagged case is scoped to a command whose declared bundle requires plan.json; the two newly un-flagged paths (absent workspace, EXECUTE by name) only remove false failures. Layer 2 in symphony-alpha keeps working against a claude-plugins with or without this change — nothing here requires the two to deploy together.

Refs ISS-5963. Layer 2 was ISS-5872 / symphony-alpha#4806.

ISS-5963 Layer 1. c3a4305 inverted the early return so a --prd run that
claims COMPLETE with no plan.json is flagged PLAN_MISSING_AT_COMPLETION.
Three verified gaps remained.

- A zero-byte plan.json was still waved through: [[ -f ]] passes on it,
  jq yields nothing, and the pendingTasks checks then read 0 pending and
  call the run clean -- so the 0-byte artifact that is the incident's own
  evidence reported success. classify_plan_artifact now treats absent,
  zero-byte, and unparseable-JSON alike: no plan was produced.
- An absent workspace failed CLOSED. "The artifact was not produced" and
  "the workspace is gone" are different facts; live-exit and boot-recovery
  reclaim the workdir right after finalization, so adjudicating a run whose
  directory has been deleted would flip a genuine success to FAILED with no
  repair path. Missing workdir now fails open.
- The predicate was the --prd proxy, so REQUEST_CHANGES -- whose result
  bundle also requires plan.json -- was invisible, and EXECUTE was excluded
  only by the accident of not carrying a PRD. run_owes_plan_json now reads
  the command: PLAN and REQUEST_CHANGES owe a plan, EXECUTE and the rest
  are excluded by name, and an unknown command falls back to the --prd
  proxy so version skew never crashes or blocks in either direction.

REQUEST_CHANGES is enforced only on the not-produced-at-all axis. The
harness seeds plan.json before an amend, so presence proves nothing there;
detecting "the amend produced nothing" needs a pre-run baseline the
detector is not given, and that AC is left open rather than guessed.

Testing: bash plugins/code/scripts/tests/test_spurious_complete.sh -- 19
pass, 0 fail. Each gap proven counterfactually by reverting its production
line: gap 1 leaves 4 red (zero-byte, malformed, whitespace-only, seeded
zero-byte all return {}), gap 2 leaves 1 red (deleted workdir reported
PLAN_MISSING_AT_COMPLETION), gap 3 leaves 4 red (REQUEST_CHANGES invisible,
EXECUTE falsely flagged). uv run pytest plugins/ green; bash -n clean.

Risks: Low, and one-sided by design -- every new flag is scoped to a
command whose declared bundle requires plan.json, and the two new
not-flagged paths (absent workspace, EXECUTE by name) only remove false
failures. The pre-existing shellcheck SC1073 in run-loop.sh is unchanged
and unrelated.
@mikeangstadt
mikeangstadt merged commit 37c48d2 into main Aug 12, 2026
5 checks passed
@mikeangstadt
mikeangstadt deleted the fix/iss-5963-spurious-complete-empty-plan-and-fail-open branch August 12, 2026 04:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant